Quickest Search Over Multiple Sequences with 

Mixed Observations 



Jun Geng Weiyu Xu Lifeng Lai 

Dept. of Electrical & Computer Engr. Dept. of Electrical & Computer Engr Dept. of Electrical & Computer Engr 
Worcester Polytechnic Institute Univ. of Iowa Worcester Polytechnic Institute 

Email: jgeng@wpi.edu Email: weiyu-xu@uiowa.edu Email: llai@wpi.edu 



o 



> 
00 

m 

O 

m 



X 



Abstract — The problem of sequentially finding an independent 
and identically distributed (i.i.d.) sequence that is drawn from a 
probability distribution Fi by searching over multiple sequences, 
some of which are drawn from Fi and the others of which are 
drawn from a different distribution Fo, is considered. The sensor 

' is allowed to take one observation at a time. It has been shown in 
a recent work that if each observation comes from one sequence, 
Cumulative Sum (CUSUM) test is optimal. In this paper, we 
propose a new approach in which each observation can be a 
linear combination of samples from multiple sequences. The test 
has two stages. In the first stage, namely scanning stage, one 
takes a linear combination of a pair of sequences with the hope 
of scanning through sequences that are unlikely to be generated 
from Fi and quickly identifying a pair of sequences such that 
at least one of them is highly likely to be generated by Fi. In 
the second stage, namely refinement stage, one examines the pair 
identified from the first stage more closely and picks one sequence 
to be the final sequence. The problem under this setup belongs 
to a class of multiple stopping time problems. In particular, it 
is an ordered two concatenated Markov stopping time problem. 
We obtain the optimal solution using the tools from the multiple 
stopping time theory. Numerical simulation results show that 
this search strategy can significantly reduce the searching time, 

, especially when Fi is rare. 

! I. Introduction 

The quickest search over multiple sequences problem, a 
generalization of the classical sequential hypothesis testing 
problem [1 1, was originally proposed in a recent paper [2 |. In 
particular, the author considered a case that multiple sequences 
are available. For each individual sequence, it may either 
, be generated by distribution Fq or Fi, and its distribution 
is independent of all other sequences. A sensor can take 
observations from these sequences, and the goal is to find 
a sequence which is generated by Fi as quickly as possible 
under an error probability constraint. Assuming that the sensor 
. can take one observation from a single sequence at a time, |2| 
' showed that the cumulative sum (CUSUM) test is optimal. 
This quickest search problem has applications in various field 
such as cognitive radio and database search. The sample 
complexity of a such search problem is analyzed in |j3]- ||4| 
studies the search problem over continuous time Brownian 
channels. The problem of recovering more than one sequence 
generated from Fi is considered in |5|. 

In this paper, we propose a new search approach: search 
with mixed observations. This search strategy consists of two 
stages. In the first stage, namely the scanning stage, the sensor 

The work of J. Geng and L. Lai was supported by the National Science 
Foundation CAREER award under grant CCF- 13- 18980 and by the National 
Science Foundation under grant DMS- 12-65663. 



takes observations that are linear combinations of samples 
from a pair of different sequences. In certain applications, such 
as cognitive radios, it is easy to obtain an observation that is 
a linear combination of signals from different sequences. The 
purpose of this stage is to scan through sequences generated 
by Fo and quickly identify a pair of sequences among which 
at least one of them is generated by Fi. In particular, if the 
sensor believes that both of the sequences that generate the 
observation are from Fq, then it discards these two sequences 
and switches to observe two new sequences. Otherwise, the 
sensor stops the scanning stage and enters the refinement stage. 
In the refinement stage, the sensor examines one of the two 
candidate sequences identified in the scanning stage carefully, 
and makes a final decision on which one of the two sequences 
is generated by Fi . Hence, in the refinement stage, no mixing 
is used anymore. 

With this mixed observation strategy, our goal is to mini- 
mize a linear combination of the searching delay and the error 
probability. Toward this goal, we need to optimize over four 
decision rules: 1) the stopping time for the scanning stage ri, 
which determines when one should stop the scanning stage 
and enter the refinement stage; 2) the sequence switching rule 
in the scanning stage </>, which determines when one should 
switch to new sequences for scanning; 3) the stopping time for 
refinement stage T2, which determines when one should stop 
the whole search process; and 4) the final decision rule in 
the refinement stage S, which determines which sequence will 
be claimed to be generated from Fi. This two stage search 
problem can be converted to an optimal multiple stopping 
time problem |6J. In particular, we show that this problem 
can be converted into an ordered two concatenated Markov 
stopping time problems. Using the optimal multiple stopping 
time theory |6|, we derive the optimal strategy for this search 
problem. We show that the optimal solutions of ri and (p turn 
out to be region rules. The optimal solution for T2 is the time 
when the error probability cost less than the future cost, and 
the optimal decision rule 6 is to pick the sequence with a larger 
posterior probability of being generated by Fi . 

The motivation to propose this mixed observation searching 
strategy is to improve the search efficiency when the presence 
of Fi is rare. If most of the sequences are generating by Fq, 
then the sensor can scan through and discard the sequences 
more quickly by this mix strategy. Our numerical results show 
that our strategy can significantly reduce the search delay when 
Fi is rare. In some sense, our strategy has a similar flavor with 
that of the group testing |7 | and compressive sensing [8| in 



which Hnear combinations of signals are observed. 

The remainder of the paper is organized as follows. The 
mathematical model is given in Section HIl Sectionlllllpresents 
the optimal solution to this quickest search problem. Numeri- 
cal examples are given in Section |IV] Finally, Section IV] offers 
concluding remarks. Due to space limitations, we omit the 
details of the proofs. 



1, 2, • • • } are i.i.d. 



n. Model 

We consider N sequences {Yi^;k 
,N, where for each i, {Y^;k - 
observations taking values in a set endowed with a cr-field 
of events, that obey one of the two hypotheses: 

Ho: Y^^Fo, fc=l,2,--- 



Y^r^Fi, fc=l,2,..-, 

where Fg and Fi are two distinct, but equivalent, distributions 
on {n,F). We use /o and /i to denote densities of Fq and 
Fi, respectively, with respect to some common dominating 
measure. The sequences for different values of i are indepen- 
dent. Moreover, whether the i*'* sequence {Y^!; k ~ 1, 2, • • • } 
is generated by Fq or Fi is independent of all other sequences. 
Here, we assume that for each i, hypothesis Hi occurs with 
prior probability tt and Hq with prior probability 1 — tt. The 
goal of the quickest search is to locate a sequence that is 
generated from Fi quickly and accurately. 

The search strategy has two stages, namely the scanning 
stage and the refinement stage. In the scanning stage, the 
sensor observes a linear combination of samples from two 
sequences, and decides whether at least one of the observed 
sequences is generated by Fi. If the sensor has enough 
confidence on that one of the observed sequences is generated 
by Fi, it enters the refinement stage, in which the sensor 
examines these two sequences identified in scanning stage 
carefully, and decides which sequence is generate by Fi. 

Specifically, in the scanning stage, at each time slot k, the 
sensor picks two sequences and s^,, and observes a linear 
combination of samples from these two sequences: 



(1) 



In this paper, we set ai = 02 = 1, which might not the 
optimal choice. We will study the optimal choice of these two 
parameters in our future work. Since each sequence has two 
possible pdfs, Zk has three possible pdfs: 

1) /o.o — /o * /o, which happens when both sequences 
s^. and s^, are generated from /q. Here * denotes 
convolution. The prior probability of this occurring is 

2) fm ~ fo * fi, which happens when one of these 
two sequences is generated from /o and the other one 
is generated from /i. The prior probability of this 
occurring is p™*^ = 27r(l — tt); 

3) fi.i — fi * fi, which happens when both sequences 
are generated from /i. The prior probability of this 
occurring is pj'^ — tt^. 

After taking observation Z^, the sensor needs to make one 
of the following three decisions: 1) to continue the scanning 



stage and to take one more observation from these two 
currently observing sequences; or 2) to continue the scanning 
stage but to take observation from two other sequences, that 
is, the sensor discards the currently observing sequences and 
switches to observe two new sequences; or 3) to stop the 
scanning stage and to enter the refinement stage to further 
examine these two candidate sequences. Hence, there are two 
decisions in the scanning stage: the stopping time ti, at which 
the sensor stops the scanning stage and enters the refinement 
stage, and the sequences switching rule cp — (0i,02,'-')' 
based on which the sensor abandons the currently observing 
sequences and switches to observe new sequences. Here, the 
element (j)k G {0, 1} denotes the sequence switching status at 
time slot k. Specifically, if ipk = 1, the sensor switches to new 
sequences, while if 0^ — 0, the sensor keeps observing the 
current two sequences. Here, the stopping time ri is adapted 
to the filtration Fk ~ (t{Zi, - ■ ■ , Zk), and the switching rule 
(j)k is a measurable function of Fk- 

In the refinement stage, we examine the two candidate 
sequences more closely. Each observation taken during the 
refinement stage will come from one sequence. Hence, at this 
stage, no mixing is used anymore. The observation sequence in 
the refinement stage is denoted as {Xj,j = 1,2,.. .}. Clearly, 
at the beginning of the refinement stage, i.e. j = 1, there is 
no difference between these two candidates s^^ and s^^, and 
hence the sensor simply picks one s° to observe: 



(2) 



After taking an observation Xj, the sensor needs to decide 
whether or not to stop the refinement stage, and if so, the 
sensor should pick one sequence from and s^^, and 
claim that it is generated from /i. Again, there are two 
decisions in this stage: the stopping time T2, at which the 
sensor decides to stop the refinement stage, and the terminal 
decision rule 6 that determines which sequence to be claimed 
as being generated by /i. T2 is adapted to the filtration 
Qj = (t{Zi, • • • , , Xi, ■ ■ ■ ,Xj). 

Two performance metrics are of interest: the total time spent 
on the search process ti + T2 and the error probability such 
that the picked sequence is generated from /q. Clearly, if we 
spend more time on the search, the error probability will be 
reduced. We aim to minimize a cost function which is a linear 
combination of these two quantities. Hence, our goal is to 
design ti, 0, T2 and 6 to solve the following optimization 
problem: 



inf {cE[ti 

Ti,0,r2,(5 



T2]+Pi{H^ ^Ho)} 



(3) 



We note that there are two inter-related stopping times in- 
volved in the problem. 

III. Solution 

In this section, we discuss the optimal solution for the 
proposed sequential search problem. We first introduce some 
important statistics used in the optimal solution. 



For the scanning stage, after taking k observations, we 
define the following posterior probabiUties: 
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As discussed in Section [III at the beginning of the scanning 



1.1 



2 mix 
~ ' Pq 



= 27r(l - tt) and pj]'" = 



stage we have 
{l-nf. 

It is easy to check that these posterior probabilities can be 
updated as follows: 



1,1 _ Pk Ji,i\^k+l) ^ Pq ji,iyz,k+i) ^ 

JZ,k{^k+l) JZ,0\^k+l) 

mix _ PT^"^ fm{Zk+l) ^ , P?r'^/m(^fc+l) ., 

JZ,k\Zk+l) JZfi\Zk+l) 
0,0 1 1,1 



1 



mix 
Pk+l' 



where 1 is the indicator function, fz,k{zk+i) and /2,o(-Zfc+i) 
are defined as 



fz,k{zk+l) — 
0.0 
Pk 

/z,o(^fc+i) 



Pk" foA^k+l) + pT''' fm{zk+l) + pI'^ fl.l{zk+l), 



Po"° fo,o{^k+l) + p^'"" fm{zk+l) + Pq^ fl.l{zk+l)- 

For the refinement stage, after taking j observations, we 
define 
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/o 



At the beginning of the refinement stage, we have 
1.1 



r, 





1,0 



0,1 



'0 ~ ' fTi 

It is easy to verify that these statistics can be updated using 
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/l(^.+l)^ 



1,1 



fi{Xj+^){r]'' + r]'') + /o(X,+i)(rf + ) ' 



fiiX,+i)r; 



1,0 



/o(^,+i)r 



A(x,+o(r^^+r:'°)+io(^+i)(^r+-r)' 



1,1 _ 1,0 
^J + 1 ''j + 1 



that 



For the brevity of notation, we further define the follow- 
ing two statistics tt^^^ = Pr |y^i''° ^ fi tt^^^ = 

Pr|Yjj''' ^ /i | . By definition, it is easy to verify 

'^ri 1,1 , 1,0 1,1 , 0,1 

In the following, we solve the multiple stopping time 
optimization problem (O. We solve this problem by decom- 
posing it into two single stopping time problems, which are 
corresponding to the refinement stage and scanning stage 
respectively. First, we have following lemma. 

Lemma 1: For any ri, and T2, the optimal decision rule 
is given as 



if TTrl^ > TTr^' 
if TTrl^ < -Kr^ 



and the corresponding cost is given as 



infPr(iJ'^ =i/o) =IE 



1 — max < TT^a^ , TT^a^ 



(4) 



(5) 



This lemma converts the cost of the error probability to a 
function of tt^^"^ and tt^^S which is a function of the refine- 



ment stage statistics v.j 



1,1 0,1 1.0 
r . r . T . 



In the following, 



we first consider the refinement stage optimization problem 
for any given ti and 4>: 



inf E [cT2 + Pr = i/o) l^rj 



(6) 



The optimal stopping time for T2 is given as: 

Lemma 2: For any given ti and 0, the optimal stopping 
time T2 is given as 

r* = inf |j > : 1 -maxivr^'STT^"! I < 



c-t-E 



in which Vr{vj) is a function that satisfies the following 
operator: 



Vr{vj) = min < 1 - max <^ tt , tt > , 



■E 



'j+i- 



We note that the form of Vr {vj ) can be obtained via an iterative 
procedure offline |9|. This lemma indicates that the optimal 
strategies in the refinement stage are related to ri , only 
through p^l^ , p™" . Hence, the minimal cost of the refinement 
stage is a function of only p^j^,p™'^: 

g - inf E [cr^ + Pr {H' = Ho) | J".,] . 

= P;(pi;\Kr72,p;r/2)- 
It is defined over the domain 

V = : < p^'^ < 1,0 < p"" < 1, 

0<pi^i+p""^ < 1}. 



Proposition 3: g is a concave function over V 

with g{l,0) = and g{0, 0) = 1. 

As the result, the original problem (|3) can be converted into 
an equivalent problem with respect to only ri and 0. Since 

inf E [c(ri + rs) + Pr {H^ = Hq)] 

Ti,(j),T2,S 

inf E [cri + E [cr2 + Pr [H^ = Hq) | J'rJ ] 

Ti,(f),T2.S 

> inf E[cri+g(pi;i,<")], 

ri,0 

the equality holds if using and 6* specified in Lemma |2] 
and [1] respectively. Therefore, ([3} is equivalent to 



inf E[cri+.g(pi;i,p™")] 



(7) 



Lemma 4: The optimal stopping rule for the scanning stage 
is given as 

Tl = inf {fc > : 5 {vWvT'') = ^ (pl'\pr^) } (8) 
and the optimal switching rule is given as 

f if <As 



1 otherwise 



(9) 



in which, Vs{-) is a function that satisfies the following 
operator 



mix 



c + mm [a, {pl'\pr''), As}} 



with 



A. 



mix 
Pk 



Vs 



{pI+v 



pk+1 



{pI+v 



Pk+i 



1,1 
Pk ^Pk 



_1,1 ^mix I 1 

Pk ^Pk ,<Pk^i 



Remark 1: Same as above, all the functions involved in this 
lemma can be computed offline. 

Remark 2: One can show that As = Ac (^Po^iP™")' hence 
it is a constant between and 1 . For this reason, we denote it 
as As rather than As (^Pfc^jP™"^ in the above lemma. 

The optimal solutions of and 0^ can be further simplified 
using the following proposition. 

Proposition 5: 1) Vs is a concave function over 

domain V, and 0<Vs < 1. 

2) Ac is a concave function over V, and < 

Ac < 1. 

Since both Vs and g (p^'^,p""^) are concave 

functions over V, Vs < over P, 

and Vs{l,0) = (?(1,0) = 0, there must exist some region, 
denoted as Rr, on which these two concave surfaces are equal 
to each other Hence, the optimal stopping time rj* can be 

described as the first hitting time of the process 
to region Rr- Similarly, Ac is a concave surface and Ag is 
a constant plane with Ag = v4c(pj'^, p™"). Hence, V can 
be divided into two connected regions R^ and V\Rij„ where 



R^ ^ {(pi'i,p™^) : <As}. Hence, the sen 

sor switches to new sequences at time slot k if 
is in R^. Hence, we have the following lemma. 

Lemma 6: There exist two regions, Rr C V and R^ C P, 
such that 



and 



\ otherwise 

IV. Simulation 



(10) 



(11) 



In this section, we give some numerical examples to il- 
lustrate the analytical results of the previous sections. In 
these numerical examples, we assume /q ^ Af{0,a'^) and 
/i ^ Af{0,P + a^). The signal-to-noise ratio is defined as 
SNR=lQ\ogP/a^. 

In the first scenario, we illustrate the cost function of the 
refinement procedure g In this simulation, we 

choose TT = 0.05, c = 0.01, a"^ ^ I and SNR = MB. 
The simulation result is shown in Figure [T] This simulation 
confirms our analysis that g is a concave function 

within [0,1] over P. We also notice that .g(l,0) = and 
(7(0, 0) = 1, this is reasonable since if the sensor knows 
both s"^ and are generated by /i, which corresponding 
to p\.'^ = 1 and p™" — 0, the sensor can make a decision on 
either of sequences without taking any observation and making 
any error, hence the cost on refinement stage is 0. Similarly, 
if the sensor knows neither nor s\_^ is generated by /i, 
that is p]:^ — and p™" — 0, no matter what decision is 
made, the cost of error would be 1. We also notice that in the 
area close to p\.'^ ~ 0,p""^ = 1, which indicates the sensor is 
quite sure that one of sequences is generated by /i, the cost 
is small. This is because the sensor can significantly reduce 
the cost of decision error by taking a few observations. 




Fig. 1: An illustration of 

In the second scenario, we illustrate the overall cost function 
Vs using the same simulation parameters. The 

simulation result is shown in Figure |2] This simulation con- 
firms that Vs (p^'^ ,p""^^ is also a concave function over P. 
Moveover, this function is flat on top since it is upper bounded 
by constant c + As- This flat area corresponds to R^, hence 

(pfc ^'P™") enters this region, the sensor would switch to 
scan new sequences at time slot k. Similarly, the cost function 



is also upper bounded by g which is shown in 

Figure [U On the region, Rr, that these two surfaces overlap 
each other, the sensor would stop the scanning stage and enter 
the refinement stage. The location of i?^ and Rr is illustrated 
in Figure |3] In this figure, the left-lower half below the blue 
line is the domain V. The region circled by the red line is the 
sequence switching region i?^, and the region circled by green 
is the scanning stop region Rr- In this simulation, Rr are two 
separate regions located around (0,1) and (1,0) respectively, 
which means the sensor will enter the refinement stage as long 
as it has enough confidence on that at least one of the observed 
sequences is generated by Fi. Rr and i?^ can be calculated 
off line. 




Fig. 2: An illusti-ation of V^ip'^^'^ 
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Fig. 3: The optimal stopping and switching regions 

In the next scenario, we illustrate the relationship between 
total cost and SNR. The total cost consists of two parts: the 
cost of searching delay and the cost of error probability . In 
the simulation, we choose tt = 0.05, c = 0.01 and = 1. 
The simulation result is shown in Figure |4] From the figure 
we can see that the curve follows a decreasing trend. This is 
consistent with our intuition, that is, the higher SNR is, the 
easier it is to distinguish between these two hypotheses. 

In the last scenario, we compare the proposed strategy with 
the strategy proposed by [2 1, which does not allow observation 
mixing and is referred as the single observation strategy 
in the sequel. We compare the search delays of these two 
strategies by keeping the error probabilities to a same level. 
In this simulation, we choose tt = 0.05. Figure |5] shows the 
simulation result. In this figure, the blue solid line is the search 
delay induced by the mix observation strategy, and the red dash 
line is the searching delay by the single observation strategy. 
As we can see, the proposed mix observation strategy saves 
about 40% of the search time since discarding two sequences 
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Fig. 4: The total cost vs. SNR under c = 0.01 and tt = 0.05 

together is more efficient than discarding sequences one by 
one. 
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Fig. 5: The comparison of delays for different searching 
strategies 

V. Conclusion 

In this paper, we have proposed a new search strategy for 
the quickest search over multiple sequences problem. We have 
formulated this problem as an optimal multiple stopping time 
problem. We have solved this problem by decomposing the 
problem into an ordered two concatenated Markov stopping 
time problem. Our simulation result shows that when iJi 
rarely occurs, the proposed strategy can significantly reduce 
the search delay. In terms of the future work, it is interesting 
and important to analytically quantify the performance gain. 
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